Recognition of handwritten digits using sparse codes generated by local feature extraction methods

We investigate when sparse coding of sensory inputs can improve performance in a classification task. For this purpose, we use a standard data set, the MNIST database of handwritten digits. We systematically study combinations of sparse coding methods and neural classifiers in a two-layer network. We find that processing the image data into a sparse code can indeed improve the classification performance, compared to directly classifying the images. Further, increasing the level of sparseness leads to even better performance, up to a point where the reduction of redundancy in the codes is offset by loss of information.